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ABSTSIACT 

Knowing  how  well  maintenance  technicians  perform  mainte¬ 
nance  on  the  job  is  necessary  in  order  to  evaluate  the  effec¬ 
tiveness  of  training.  This  paper  reviews  data  on  one  possible 
measure;  specifically,  the  unnecessary  removal  of  non-faulty 
parts  during  actions  taken  to  identify  and  correct  malfunc¬ 
tions  in  equipment.  Such  data  may  be  found  in  the  maintenance 
management  data  systems  of  the  military  services. 

It  was  found  that  non-faulty  components  are  removed  in  4 
to  43  percent  of  all  corrective  maintenance  actions  and  account 
for  9  to  32  percent  of  all  maintenance  man-hours.  Technicians 
fail  to  find  a  faulty  part  or  damage  a  good  part  in  about  10 
percent  of  all  corrective  maintenance  actions. 

These  findings  may  be  due  to  inadequate  test  equipment, 
tools,  and  maintenance  manuals,  as  well  as  to  inadequate  train¬ 
ing. 

There  is  a  need  to  collect  data  on  the  performance  of  main 
tenance  technicians  on  the  job  in  a  way  that  can  be  related  sys 
tematically  to  procedures  used  in  military  selection  and  train- 
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SUMMARY 


A.  PURPOSE 

The  purpose  of  this  paper  is  to  review  data  that  describe 
the  job  performance  of  military  maintenance  technicians. 

B .  BACKGROUND 

The  effectiveness  of  military  maintenance  training  is 
almost  always  evaluated  on  the  basis  of  how  well  students  per¬ 
form  at  school,  i.e.,  test  scores  at  the  completion  of  a  course. 
An  important  question,  however,  is  how  well  training  at  school 
prepares  maintenance  technicians  to  perform  maintenance  on  the 
job.  Little  objective  data  are  available  on  the  job  performance 
of  maintenance  technicians.  Without  such  data,  it  is  difficult 
to  assess  the  effectiveness  of  maintenance  training,  an  impor¬ 
tant  issue  in  cost-effectiveness  evaluations  of  military  train¬ 
ing. 

Each  military  service  operates  a  maintenance  management 
data  system  that  contains  information  on  the  conduct  of  all 
maintenance  actions  (or  tasks),  e.g.,  what  equipment  was  main¬ 
tained,  why  maintenance  was  required,  what  was  done  and  who  did 
it.  Although  these  systems  were  not  designed  to  answer  questions 
about  training,  the  present  effort  was  an  attempt  to  see  wnether 
they  might  be  used,  in  some  way,  for  such  purposes.  The  scope 
of  the  effort  was  limited  to  data  on  the  unnecessary  removal 
of  good  parts  during  actions  taken  to  identify  and  correct  mal¬ 
functions  in  equipment. 


S-l 


C.  FINDINGS 

According  to  seven  studies,  non-faulty  parts  were  removed 
in  4  to  43  percent  of  all  corrective  maintenance  actions,  and 
account  for  9  to  32  percent  of  all  maintenance  man-hours.  One 
study  reports  that  technicians  failed  to  find  a  faulty  part  or 
damaged  a  good  part  in  10  percent  of  all  maintenance  actions. 

Only  limited  efforts  were  made  in  these  studies  to  examine 
why  these  amounts  of  ineffective  maintenance  were  observed. 
Suggestions  are  offered  that  the  reasons  include  inadequate 
test  equipment,  tools,  and  documentation,  as  well  as  inadequate 
training. 

D.  CONCLUSIONS 

Available  data,  though  limited  in  scope  primarily  to  the 
unnecessary  removal  of  non-faulty  parts,  offer  strong  evidence 
that  maintenance  technicians  may  conduct  maintenance  in  an 
inappropriate  and  inefficient  manner. 

E .  RECOMMENDATIONS 

It  is  recommended  that  additional  data  be  collected  on  the 
performance  of  maintenance  technicians  on  the  job  to  estimate 
not  only  the  unnecessary  removal  of  non-faulty  parts  but  also 
the  failure  to  remove  faulty  parts  and  damage  caused  to  good 
parts  while  performing  maintenance.  Data  are  also  needed  to 
identify  the  factors  that  may  lead  to  inappropriate  maintenance, 
such  as  inappropriate  test  equipment,  tools,  documentation,  and 
training  among  a  number  of  possible  factors. 


5-2 


t 


I .  INTRODUCTION 

It  is  obvious  that  the  performance  of  maintenance  techni¬ 
cians  is  one  of  the  factors  that  can  influence  the  operational 
readiness  of  weapon  systems  in  the  field.  Nevertheless,  sur¬ 
prisingly  little  objective  data  are  available  to  document  how 
well  technicians  do  what  they  are  supposed  to  do.  This  paper 
summarizes  the  objective  data  that  we  were  able  to  compile 
concerning  the  job  performance  of  maintenance  technicians.* 
Objective  job  performance  data  are  needed  to  evaluate  the 
effectiveness  of  procedures  used  by  the  military  services  to 
select  and  train  maintenance  technicians.  At  present,  methods 
of  selection  and  training  are  validated  almost  entirely  on  the 
basis  of  how  maintenance  technicians  perform  at  school  rather 
than  on  the  job.  Supervisors'  ratings  are  sometimes  used  to 
evaluate  training  courses.  This  method  of  validation  involves 
subjective  judgments  that  may  be  influenced  by  impressions 
about  motivation  and  cooperation  that  have  little  to  do  with 
capability  to  perform  well  on  the  job. 


*This  study  was  performed  for  the  Office  of  the  Deputy  Under 
Secretary  of  Defense  for  Research  and  Engineering  (Research 
and  Advanced  Technology) ,  under  the  technical  cognizance  of 
the  Military  Assistant  for  Training  and  Personnel  Technology. 
It  is  one  of  a  series  concerned  with  Cost  and  Effectiveness 
Methods  for  Defense  Training  (DARPA  TO  T-134). 
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II.  MAINTENANCE  MANAGEMENT  DATA  SYSTEMS 


The  military  services  operate  large  data  management  systems 
to  provide  detailed  information  on  the  current  maintenance  status 
of  military  equipment.  These  data  systems  are  identified  in 
Table  1.  The  general  purpose  of  these  systems  is  to  provide 
information  needed  to  manage  the  maintenance  of  weapons  and  sup¬ 
port  equipment,  the  availability  of  spare  parts,  the  types  of 
malfunctions  that  are  being  encountered,  and  so  on.  These  sys¬ 
tems  were  designed  to  provide  information  needed  for  purposes 
of  maintenance  and  logistics  and  not  about  the  performance  of 
military  technicians. 

TABLE  1.  MAINTENANCE  MANAGEMENT  DATA  SYSTEMS  USED 
BY  THE  MILITARY  SERVICES 


Service 


Maintenance  Management  System 
Name  S 


Short  Title 


Army 


The  Army  Maintenance  Management  System  TAMMS 


Navy 


Naval  Ships'  Maintenance  and  Material 
Management  System 


Ships'  3-M 


Navy 


Naval  Aviation  Maintenance  and 
Material  Management  System 


Aviation  3-M 


Air  Force  Air  Force  Maintenance  Management  Systems  66-1  and  66-5 
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The  possibility  of  using  data  available  in  these  systems 
to  describe  the  performance  of  maintenance  technicians  in  the 
field  has  been  examined  (see  String  and  Orlansky,  1981).  As 
presently  constituted,  these  systems  cannot  provide  information 
useful  for  assessing  the  on-the-job  effectiveness  of  alternative 
methods  of  selection  and  training.  The  ability  to  identify  and 
track  individuals  is  a  mandatory  requirement  of  any  attempt  to 
relate  criteria  for  selection  or  method  of  training  to  perform¬ 
ance  on  the  job.  The  names  of  individuals  who  perform  mainte¬ 
nance  actions  are  not  kept  in  the  permanent  records  maintained 
in  the  central  data  files  of  each  service.  Maintenance  records, 
which  include  names  of  personnel  who  did  the  work,  are  kept 
only  at  the  field  activities,  but  they  are  discarded  after  6 
months.  The  use  of  maintenance  records  with  personal  identifi¬ 
cation  for  analytical  purposes  would  require  special  methods  of 
processing  in  order  not  to  infringe  on  provisions  of  the  Privacy 
Act.  Even  so,  such  records  are  not  precise  enough'  to  distinguish 
what  parts  of  a  maintenance  action  were  performed  by  a  particular 
individual,  especially  when  the  work  is  performed  over  more  than 
one  shift.  The  practice  of  cross-skill  maintenance,  that  is, 
to  train  individuals  to  maintain  a  wide  variety  of  equipment 
under  combat  conditions,  assigns  individuals  to  tasks  for  which 
they  were  not  trained  at  school;  this  practice  would  complicate 
any  analytical  effort  to  relate  training  at  school  to  performance 
on  the  job.  In  brief,  it  was  concluded  that  presently  available 
maintenance  data  records  cannot  be  used  to  assess  the  effective¬ 
ness  on  the  job  of  various  methods  of  training  at  school.  It 
is  conceivable  that  these  systems  could  be  modified  to  provide 
the  data  that  would  be  needed,  but  that  is  not  the  subject  of 
this  paper. 


III.  DATA  ON  INCORRECT  MAINTENANCE  ATTRIBUTED  TO  PERSONNEL 


Certain  information  collected  by  the  maintenance  management 
data  systems  may  be  used  to  make  inferences  about  the  quality 
of  performance  of  maintenance  technicians  in  the  actual  environ¬ 
ment  of  day-to-day  work.  Specific  examples  concern  data  on 
components  removed  erroneously,  i.e.,  components  removed  for 
replacement  or  repair  that  were  found  later  not  to  contain  any 
malfunction;  another  would  be  a  report  of  no  malfunction  when 
one  was  found  immediately  afterwards.  Such  data  may  be  used  to 
characterize  the  work  of  a  group  of  technicians  in  a  particular 
work  center;  it  does  not  identify  particular  individuals  and 
therefore  cannot  be  related  to  their  individual  characteristics 
with  respect  to  test  performance  when  selected  or  method  of 
training  prior  to  their  current  assignment.  Some  qualifications 
about  the  use  of  data  on  the  removal  of  non-faulty  parts  will 
be  discussed  later. 

A.  NAVY  F-14A  AIRCRAFT 

According  to  Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980) 
maintenance  technicians  can  produce  three  kinds  of  errors  in 
organizational  maintenance:  replace  a  good  unit,  fail  to  replace 
a  bad  unit,  or  damage  the  system  in  some  way  (see  Table  2). 

Some  of  these  errors  can  produce  significant  effects,  e.g., 
abort  an  operation,  require  repetition  of  the  troubleshooting 
and  repair  activity,  waste  spare  parts,  place  an  additional 
load  on  the  maintenance  activity,  or  perhaps  lead  to  an  injury 
or  accident. 
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TABLE  2.  KINDS  OF  MAINTENANCE  ERRORS 


Type  of 

Error  _ Explanation  of  Error 


Typical 

Source  of  Error 


I  Removal  of  functioning  equipment:  Troubleshooting 

technician  replaces  a  unit  that 
has  not  malfunctioned. 


II  Failure  to  remove  faulty  equip-  Troubleshooting; 

ment:  technician  fails  to  recog-  checkout 

nize  a  unit  that  has  malfunctioned 
or  has  been  improperly  handled. 


d 


Damage  to  equipment:  technician 
fails  to  accomplish  a  corrective 
or  preventive  action  properly. 


Removal/ instal¬ 
lation;  service; 
repair;  adjust/ 
align. 


Source:  Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980),  p.  12 


Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980)  used  data  from 
the  Naval  Aviation  Maintenance  and  Material  Management  System 
(Aviation  3-M)  to  describe  the  readiness  status  of  six  F-14A 
squadrons  (72  aircraft)  over  a  period  of  1  year  (Table  3).  At 
any  given  time,  5.09  (42  percent)  of  the  12  aircraft  in  each 
squadron  were  not  ready  for  operational  assignments,  for  main¬ 
tenance  conditions  given  in  the  table;  two  (17  percent)  of  the 
aircraft  were  not  ready  because  of  unscheduled  maintenance. 

Each  F-14A  aircraft  required  an  average  of  43  man-hours  of  or¬ 
ganizational  level  maintenance  for  each  hour  of  flight  (Table  4) 
over  16  (38  percent)  of  these  man-hours  were  devoted  to  correc¬ 
tive  maintenance. 

An  analysis  was  made  of  the  frequency  of  each  type  of  cor¬ 
rective  maintenance  (CM)  error  according  to  responsible  work 
center,  Table  5.  Nearly  14  percent  of  remove  and  replace  opera¬ 
tions  (4  percent  of  all  CM  actions)  involved  removals  of  func¬ 
tioning  equipment  (a  Type  I  error);  nearly  10  percent  of  all  CM 
actions  failed  to  remove  faulty  equipment  (a  Type  II  error)  or 
resulted  in  damage  to  equipment  (a  Type  d  error).  Nearly  14  per 
cent  of  all  CM  actions  resulted  in  one  of  these  error  conditions 
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TABLE  3.  READINESS  CONDITION  OF  AN  AVERAGE  SQUADRON 
OF  F-14A  AIRCRAFT  OVER  ONE  YEAR 
(12  aircraft  per  squadron) 


Operational 
Status  of 
Aircraft 

Maintenance 

Condition 

Average  Number 
of  aircraft 

Number 

Percent 

Ready 

Full 

systems  capable 

6.20 

51% 

Reduced  material  condition 
due  to  unscheduled  maintenance 

.38 

3 

Not 

fully  equipped 

.39 

3 

Total 

6.97 

58% 

Not  ready 

Due 

to  scheduled  maintenance 

.48 

4% 

Due 

to  unscheduled  maintenance 

2.00 

17 

Due 

to  supply 

2.61 

22 

Total 

5.09 

42% 

TOTAL 

12.06 

100% 

SOURCE:  Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980) 

p.  70 

TABLE  4. 

ORGANIZATIONAL  MAINTENANCE  MAN-HOURS  PER  FLIGHT 
HOUR  IN  SIX  SQUADRONS  OF  F-14A  AIRCRAFT 

Maintenance  Category  Man- 

hours  Per 

Flight  Hour 

Number 

Percent 

Planned  maintenance  (PM) 

19.2 

44% 

Corrective 

maintenance  (CM) 

16.4 

38 

Support  actions 

(SAF) 

6.4 

15 

Technical  directive  compliance  (TDC ) 

1.2 

3 

TOTAL 

43.2 

100% 

SOURCE:  Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980),  p.  71 
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TABLE  5.  SUMMARY  OF  CORRECTIVE  MAINTENANCE  ERRORS  FOR  F-14A  AIRCRAFT 


Type  II/d“'e  Errors'5 


Work  Center 


Organizational  Main¬ 
tenance  Department 

Rawer  Plants  (110) 

Airframe  (120) 

Gorrosion  Control  (121) 

Aviator  Equipment  (131) 

Safety  Equipment  (132) 

Electronics  (210) 

Electrical  Instruments 
(220) 

Armaments  (230) 

Electro-Weapons  Control 
( 232)c 

Troubleshooters  (320) 


aRemoval  of  non-faulty  parts 

^Failure  to  recognize  a  malfunction/damage  induced  by  technician 
cIncludes  AN/AW3-9  Radar  that  accounted  for  over  60  percent  of  CM 
actions  and  errors 
^Expressed  as  percent  of  RR  jobs 
Expressed  as  percent  of  all  CM  jobs 

All  CM  actions  are  the  sum  of:  TS  +  RIP  +  RR  +  CANN 
where:  TS  =  Total  Troubleshoot  Jobs 

RIP  =  Total  Repair-In-Place  Jobs 

RR  =  Total  Remove-and-Replace  Jobs 

CANN  =  Total  Cannibalization  Jobs 


All  Errors  are  the  sum  of:  El  +  El ID 
where:  El  =  Total  Errors  of  Type  I 

EIID  =  Total  Errors  of  Type  II  or  d 

SOURCE:  Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980),  p.  74 

Private  conversations  with  David  Gold  and  Sal  Ravo,  XYZYX  Informa 
tion  Corporation. 


The  Aviation  3-M  system  does  not  report  directly  that  tech¬ 
nicians  have  produced  various  types  of  errors;  this  is  inferred 
by  Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980)  in  the  Tables 
shown  above.  In  the  Aviation  3-M  system,  the  Aircraft  Interme¬ 
diate  Maintenance  Department  Report  cites  cases  where  "no  de¬ 
fects"  were  found  in  components  removed  from  aircraft.  Gold, 
Kleine,  Fuchs,  Ravo,  and  Inaba  (1980)  call  these  "unjustified 
removals"  (Type  1  error).  This  interpretation  does  not  seem 
appropriate  in  all  cases.  Consider  the  following  situations: 

(1)  a  technician  removed  a  component  because  the  test  equipment 
available  to  him  was  not  capable  of  isolating  a  malfunction  to 
a  single  system  element  (i.e.,  it  could  localize  the  fault  to  a 
group  of  black  boxes  of  which  only  one  was  found  later  to  be 
faulty);  (2)  two  elements  of  a  system,  one  highly  interactive 
so  the  one  element  will'  function  correctly  with  one  article 
(black  box)  of  the  second  element  but  not  with  a  second  black 
box  of  the  same  model.  When  under  pressure  to  meet  a  required 
flight  time,  a  technician  may  knowingly  replace  a  number  of 
black  boxes,  without  testing,  to  be  certain  that  the  faulty 
one  is  replaced  prior  to  flight  time. 

Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba  (1980)  also  inferred 
failures  to  remove  faulty  equipment  or  faulty  repair  (Type  II 
and  Type  d  errors)  in  cases  where  their  analyses  related  observed 
malfunctions  to  previous  reports  of  failure  to  find  any  malfunc¬ 
tion  in  the  same  components.  Some  of  these  cases  may  have  in¬ 
volved  temporary  "auick-f ixes  ,"  e.g.,  tightening  a  connector  or 
fastener  known  to  be  degraded.  As  noted  elsewhere  in  this  paper, 
the  Aviation  3-M  maintenance  data  system  was  designed  to  provide 
information  on  the  status  of  equipment  and  not  on  the  quality  of 
human  performance  as  a  possible  source  of  certain  malfunctions. 
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B.  ARMY  RECONNAISSANCE  VEHICLE  TURRETS 

A  recent  study  of  organizational  level  support  in  an  Army 
brigade-sized  unit  produced  similar  results.  In  the  Army,  parts 
found  to  be  faulty  during  organizational  maintenance  are  submit¬ 
ted  for  exchange  to  a  shop  which  performs  direct  support  main¬ 
tenance.  A  Maintenance  Request  Form  (DA  2407)  is  filled  out  for 
each  exchange.  Dressel  and  Shields  (1979)  determined  whether 
the  parts  submitted  for  repair  over  a  period  of  1  year  were 
found  later  to  be  faulty;  attention  was  limited  to  the  turret 
of  the  Armored  Reconnaisance  Airborne  Assault  Vehicle  (M  551). 

On  behalf  of  the  study,  the  maintenance  shop  manager  completed 
a  special  form  ( ARI  SF  77-1)  which  recorded  the  specific  repairs 
required  and  other  information  of  interest  for  each  part  that 
was  exchanged.  The  main  findings  are  summarized  in  Table  6. 
Almost  half  (42  percent)  of  the  items  submitted  for  repair  were 
not  faulty;  32  percent  of  all  man-hours  spent  in  the  repair 
Shop  were  applied  to  items  found  not  to  contain  a  fault. 

TABLE  6.  ANALYSIS  OF  ITEMS  REMOVED  FOR  MAINTENANCE  PURPOSES 
FROM  TURRETS  OF  THE  ARMORED  RECONNAISSANCE  AIRBORNE  ASSAULT 
VEHICLE  IN  AN  ARMY  BRIGADE  FOR  1  YEAR 
(Data  from  Dressel  and  Shields,  1979) 


Data  On  Removals 

Total 

Non-Faulty 

Items 

Number 

Percent 

Requests  for  repair 

584 

246 

42% 

Repair  time 

1146  hrs 

367  hrs 

32 

Average  time  in  shop 

5.6  days 

4.0  days 

71 

Average  repair  time3 

2.3  hrs 

1.5  hrs 

65 

Cost  of  items  submitted 
for  repair 

$1.24  M 

$0.36  M 

29 

aConfirmed  malfunctions  only 


C.  NAVY  EA-6B,  E-2C,  SH-3H,  AND  S-3A  AIRCRAFT 


Jewell  and  Webman  ( 1979)  analyzed  maintenance  records  on 
all  Navy  EA-6B,  E-2C,  SH-3H,  and  S-3A  aircraft  for  1977  as 
reported  in  the  Naval  Aviation  Maintenance  and  Material  Manage¬ 
ment  System  (Aviation  3-M) .  This  data  base  accounted  for  a 
total  of  about  1.8  million  man-hours  of  maintenance  work  and 
about  385,000  maintenance  actions.  Attention  was  given  primar¬ 
ily  to  "no-defect  maintenance"  defined  as  unscheduled  mainte¬ 
nance  on  components  for  which  no  corrective  action  was  required 
(See  Table  7).  About  15  percent  of  all  actions  both  in  organi¬ 
zational  and  intermediate  maintenance  were  on  items  found  not 
to  have  any  defects;  about  17  percent  of  the  man-hours  in  orga¬ 
nizational  maintenance  and  about  9  percent  of  the  man-hours  in 
intermediate  maintenance  were  expended  on  items  found  not  to 
have  any  defects.  On  the  basis  of  interviews  with  maintenance 
personnel,  Jewell  and  Webman  (1979)  conclude  that  maintenance 
is  performed  on  items  found  not  to  have  defects  because  of 
inadequate  built-in  test  equipment  that  cannot  isolate  equipment 


TABLE  7.  AMOUNT  OF  MAINTENANCE  ACTIVITIES  ON  NAVAL  AIRCRAFT 
WHERE  NO  DEFECT  WAS  FOUND,  1977 
[Source:  Jewell  and  Webman  (1979)] 


Maintenance  Activities 

,  1977a 

Leval  of 

Man-Hours 

Maintenance  Actions 

Maintenance 

Total 

(000) 

Percentage  due 
to  Removal  of 
No-Defect  Items 

Total 

(000) 

Percentage  due 
to  Removal  of 
No-Defect  Items 

Organizational 

1119 

17.0% 

322 

14.8% 

Intermediate 

650 

9.3 

63 

16.1 

aData  for  maintenance  on  all  Navy  EA-6B,  E-2C,  SH-3H,  and  S-3A 
aircraft  during  1977  as  shown  in  Naval  Aviation  Maintenance  and 
Material  Management  System. 
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test  equipment  that  cannot  isolate  equipment  failure,  because 
system  complexity  precludes  simple  fault  isolation,  and  because 
systems  integration  requires  personnel  to  be  knowledgeable  in 
several  systems  without  adequate  training  on  all  of  them.  These 
explanations  are  based  on  the  interviews;  although  they  appear 
reasonable,  no  data  are  offered  to  support  these  conclusions. 

D.  ARMY  ELECTRICAL  AND  VEHICULAR  COMPONENTS 

"In  a  test  conducted  at  Fort  Carson,  Colorado,  an  average 
of  35  percent  of  the  generators,  regulators,  alternators,  dis¬ 
tributors,  and  starters  returned  as  unusable  were  actually 
serviceable".*  According  to  the  Brown  Board  Survey  1966,  43 
percent  of  the  vehicular  components  removed  as  faulty  in  field 
maintenance  were  found  later  to  be  good. 

E.  AIR  FORCE  A-7D,  F-111A,  AND  F-4D  AIRCRAFT 

Johnson  and  Reel  (1973)  report  that  9  to  13  percent  of 
the  components  removed  for  failure  on  three  types  of  aircraft 
( A-7D ,  F-111A,  and  F-4D)  were  found  later  in  the  shop  to  be 
serviceable;  note  that  these  data  refer  to  percent  of  components 
removed  that  were  found  to  be  good  rather  than  percent  of 
maintenance  actions  in  which  good  parts  were  removed.  Johnson 
and  Reel  also  report  that  85  percent  of  the  good  parts  removed 
came  from  avionics  systems;  the  remainder  came  from  airframe 
and  utility  systems,  propulsion,  instruments,  and  autopilots. 


*This  statement  appears  in  Buchan  and  Knutson,  1977.  The  study, 
Troubleshooting  Test  Conducted  by  the  USAMMCS  at  Fort  Carson, 
CO,  1-31  July  1974,  was  not  seen.  A  relevant  table  from  that 
study  and  information  about  the  Brown  Board  Survey  were  pro¬ 
vided  by  J.  Shields  of  the  Army  Research  Institute. 


* 
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F.  ARMY  UH-1H,  CH-47C  ,  AND  CH-57B  HELICOPTERS 


The  maintenance  of  Army  helicopters  was  analyzed  because 
"over  50%  of  Army  aircraft  maintenance  diagnoses  at  organiza¬ 
tional  level  were  reported  as  being  incorrect  by  a  high-ranking 
military  official.  U.S.  Army  Agency  for  Aviation  Safety  (USAAAVS) 
reported  that  poor  inspections  and/or  improperly  performed  main¬ 
tenance  actions  were  frequently  the  cause  of  helicopter  mishaps" 
(Holbert  and  Newport,  1975,  p.  24). 

An  analysis  of  over  5000  maintenance  records  of  the  UH-1H 
and  CH-47C  helicopters  (6  months  each)  and  of  maintenance  rec¬ 
ords  of  8500  flight  hours  for  the  CH-54B  helicopter  (30  months) 
showed  that  there  were  0.23,  0.32,  and  0.07  repetitive  mainte¬ 
nance  actions  (respectively)  per  flight  hour;  these  maintenance 
actions  apply  to  the  same  malfunctions  on  the  same  aircraft 
reported  frequently  over  short  time  periods.  Total  maintenance 
actions,  including  those  for  non-repetitive  malfunctions,  were 
not  reported.  The  repetitive  actions  identify  frequently  recur¬ 
ring  malfunctions,  e.g.,  altitude  indicator,  high  engine  oil 
temperature,  fuel-low  light,  engine  exhaust  duct.  About  half 
of  the  repetitive  maintenance  actions  are  attributed  to  inade¬ 
quate  test  equipment,  troubleshooting,  and  standard  maintenance’ 
practices;  about  20  percent  are  attributed  to  inadequate  train¬ 
ing,  tools,  and  maintenance  manuals. 

Holbert  and  Newport  tried  to  determine  the  frequency  of 
incorrect  diagnoses  of  malfunctions  by  comparing  records  sub¬ 
mitted  by  the  organizational-level  maintenance  activity  with 
those  for  the  same  components  at  depot-level  overhaul  and 
inspection.  Records  of  maintenance  of  the  UH-1H  and  CH-47C 
helicopters  at  three  operational  bases  and  two  depots  for  a 
6-month  period  were  examined.  It  was  found,  that  depots  are 
not  required  to  use  the  organizational-level  form  (DA  Form  2410) 
with  diagnostic  information  if  they  use  assembly-line  produc¬ 
tion  methods  for  overhaul.  Therefore,  an  objective  comparison 
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of  diagnoses  of  malfunctions  could  not  be  made.  On  the  basis 
of  interviews  with  personnel  at  the  depots,  it  was  estimated 
that  15  to  25  percent  of  transmissions  going  through  overhaul 
are  found  to  have  no  defects  or  malfunctions. 

G.  ARMY  HELICOPTERS 

Reilly  (1977)  reviewed  reports  of  13,037  mishaps  to  six 
classes  of  Army  helicopters  from  1969  to  1976.  The  overall 
proportion  of  mishaps  attributed  to  maintenance  error  was  5.7 
percent.  Most  errors  were  attributed  to  factors  other  than 
errors  in  maintenance,  such  as  materiel  malfunction  (52  percent) 
and  crew  error  (29  percent);  the  total  cost  of  all  mishaps  in 
this  sample  was  $270  million.  Since  this  study  does  not  report 
information  on  the  removal  of  non-faulty  parts,  it  is  not 
included  in  the  discussion  that  follows. 


IV.  DISCUSSION 


A  summary  of  these  studies  of  the  removal  of  non-faulty 
parts  during  corrective  maintenance  appears  in  Table  8.  The 
removal  of  non-faulty  parts  occurs  in  4  to  43  percent  of  all 
corrective  maintenance  actions  in  these  data;  the  median  value 
of  11  data  sets  is  15  percent.  The  removal  of  non-faulty  parts 
accounts  for  9  to  32  percent  of  all  maintenance  man-hours  (for 
three  cases  where  such  data  were  reported).  According  to  one 
study,  technicians  fail  to  find  a  faulty  part  or  damage  a  good 
part  in  about  10  percent  of  all  corrective  maintenance  actions 
(Gold,  Kleine,  Fuchs,  Ravo,  and  Inaba,  1980). 

These  data  suggest  that  inadequate  performance  by  techni¬ 
cians  is  a  factor  that  contributes  to  the  "not-ready"  status. of 
military  equipment.  Other  factors  would  include  the  unavaila¬ 
bility  of  spare  parts,  test  equipment,  and  up-to-date  technical 
documentation.  For  example,  Gold,  Kleine,  Fuchs,  Ravo,  and 
Inaba  (1980)  estimate  that  over  a  1-year  period,  an  average  of 
22  percent  of  the  F-14A  aircraft  were  not  ready  for  reasons 
due  to  supply.  According  to  a  questionnaire,  about  50  percent 
of  551  Army  technicians  believed  that  repetitive  maintenance 
(same  malfunction)  of  Army  helicopters  was  due  primarily  to 
inadequate  test  equipment,  troubleshooting,  and  standard  main¬ 
tenance  practices;  about  20  percent  gave  inadequate  training, 
tools,  and  maintenance  manuals  as  a  secondary  cause  (Holbert 
and  Newport,  1975).  These  findings  appear  to  identify  a  signif¬ 
icant  problem  in  military  maintenance  but  do  not  suggest  a 
means  to  its  solution. 
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The  data  sample  is  small  and  may  not  be  representative. 

The  removal  of  non-faulty  parts  may  not  always  be  an  inappro¬ 
priate  action,  e.g.,  the  test  equipment  may  malfunction  or  not 
be  capable  of  distinguishing  between  a  faulty  and  non-faulty 
part;  if  the  technician  is  under  pressure  to  have  equipment 
ready  for  a  mission,  he  may  remove  and  replace  a  large  number 
of  components  without  sufficient  tests  in  order  to  make  sure 
that  all  possible  malfunctions  have  been  removed.  Finally, 
the  data  apply  to  all  maintenance  actions  within  a  large  unit 
and  not  to  the  performance  of  particular  individuals. 

One  particular  value  of  data  describing  the  quality  of 
performance  of  maintenance  personnel  on  jobs  in  operational 
settings  would  be  their  use  in  validating  selection  standards 
for  recruiting  and  assigning  to  career  paths  and  evaluating 
the  effectiveness  of  various  methods  of  training  (e.g..,  conven¬ 
tional  instruction  compared  to  computer-based  instruction,  use 
of  maintenance  training  simulators  as  opposed  to  actual  equip¬ 
ment  training).  As  a  general  matter,  the  effectiveness  of 
military  selection  and  training  has  been  evaluated  on  the  basis 
of  performance  of  technicians  at  school  and  not  on  the  job. 

The  latter  is  the  more  relevant  criterion. 

It  is  conceivable  that  the  data  generated  through  mainte¬ 
nance  management  systems  of  the  military  services  could  be 
modified  to  provide  information  on  the  performance  of  mainte¬ 
nance  technicians.  These  systems  were  designed  primarily  to 
manage  maintenance  services  and  cannot  be  faulted  for  not  pro¬ 
viding  information  about  personnel  relevant  to  selection  and 
training.  A  prototype  system  for  providing  some  of  this  infor¬ 
mation  has  been  developed  and  is  now  being  tested  by  the  U.S. 
Army  Research  Institute  (Katz  and  Drillings,  1981).  Called 
the  Army  Maintenance  Performance  System,  it  records  the  work 
experience  (time  on  each  technical  task  in  the  maintenance 
battalion)  and  training  (courses  and  qualification  tests)  of 
each  maintenance  technician.  This  record  system  is  not  planned 
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to  be  part  of  The  Army  Maintenance  Management  System;  it  would 
be  used  by  work  supervisors  and  training  managers;  each  soldier 
would  carry  his  own  record  of  experience  and  skill  history.  It 
does  pot  appear  that  this  record  system  would  contain  informa¬ 
tion  about  effective  and  ineffective  performance,  e.g.,  time  to 
diagnose  malfunctions,  success  and  failure  to  diagnose  malfunc¬ 
tions  of  various  types. 
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V.  FINDINGS 

Listed  below  are  the  major  findings  resulting  from  this 
study . 

1.  Non- faulty  components  are  removed  in  4  to  43  percent 
of  all  corrective  maintenance  actions  and  account  for 
9  to  32  percent  of  all  maintenance  man-hours. 

2.  Technicians  fail  to  find  a  faulty  part  or  damage  a 
good  part  in  about  10  percent  of  all  corrective  main¬ 
tenance  actions. 

3.  Maintenance  technicians  believe  that  repetitive  main¬ 
tenance  for  the  same  malfunction  is  due  primarily  to 
inadequate  test  equipment,  troubleshooting,  and  stand¬ 
ard  maintenance  practices,  and  secondarily  to  inade¬ 
quate  training,  tools,  and  maintenance  manuals. 

4.  These  findings  are  based  on  seven  studies  reported 
from  1975  to  1980.  Diagnostic  studies  are  needed  to 
clarify  the  extent  to  which  human  performance  affects 
the  quality  of  maintenance  in  different  types  of  weapon 
and  support  systems  and  to  identify  ways  of  improving 
the  personnel  aspects  of  maintenance. 

5.  Data  on  the  performance  of  maintenance  technicians  on 
the  job  should  be  collected  in  a  way  that  can  be  re¬ 
lated  to  procedures  used  in  military  selection  and 
training. 
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